Offline-to-Online Knowledge Distillation for Video Instance Segmentation

In this paper, we present offline-to-online knowledge distillation (OOKD) for video instance segmentation (VIS), which transfers a wealth of video knowledge from an offline model to an online model for consistent prediction. Unlike previous methods that having adopting either an online or offline model, our single online model takes advantage of both models by distilling offline knowledge. To transfer knowledge correctly, we propose query filtering and association (QFA), which filters irrelevant queries to exact instances. Our KD with QFA increases the robustness of feature matching by encoding object-centric features from a single frame supplemented by long-range global information. We also propose a simple data augmentation scheme for knowledge distillation in the VIS task that fairly transfers the knowledge of all classes into the online model. Extensive experiments show that our method significantly improves the performance in video instance segmentation, especially for challenging datasets including long, dynamic sequences. Our method also achieves state-of-the-art performance on YTVIS-21, YTVIS-22, and OVIS datasets, with mAP scores of 46.1%, 43.6%, and 31.1%, respectively

Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency

We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision. Our technical contributions are three-fold. First, we highlight the fundamental difference between inverse and forward projection while modeling the individual motion of each rigid object, and propose a geometrically correct projection pipeline using a neural forward projection module. Second, we design a unified instance-aware photometric and geometric consistency loss that holistically imposes self-supervisory signals for every background and object region. Lastly, we introduce a general-purpose auto-annotation scheme using any off-the-shelf instance segmentation and optical flow models to produce video instance segmentation maps that will be utilized as input to our training pipeline. These proposed elements are validated in a detailed ablation study. Through extensive experiments conducted on the KITTI and Cityscapes dataset, our framework is shown to outperform the state-of-the-art depth and motion estimation methods. Our code, dataset, and models are available at https://github.com/SeokjuLee/Insta-DM .Comment: Accepted to AAAI 2021. Code/dataset/models are available at https://github.com/SeokjuLee/Insta-DM. arXiv admin note: substantial text overlap with arXiv:1912.0935

Implicit Neural Image Stitching With Enhanced and Blended Feature Reconstruction

Existing frameworks for image stitching often provide visually reasonable stitchings. However, they suffer from blurry artifacts and disparities in illumination, depth level, etc. Although the recent learning-based stitchings relax such disparities, the required methods impose sacrifice of image qualities failing to capture high-frequency details for stitched images. To address the problem, we propose a novel approach, implicit Neural Image Stitching (NIS) that extends arbitrary-scale super-resolution. Our method estimates Fourier coefficients of images for quality-enhancing warps. Then, the suggested model blends color mismatches and misalignment in the latent space and decodes the features into RGB values of stitched images. Our experiments show that our approach achieves improvement in resolving the low-definition imaging of the previous deep image stitching with favorable accelerated image-enhancing methods. Our source code is available at https://github.com/minshu-kim/NIS

Foraging Behavior and Diet of Northward Migrating Bar-Tailed Godwits (Limosa laponica) and Great Knots (Calidris tenuirostris) at a Key Stopover Site

After the completion of the Saemangeum reclamation, the Geum Estuary has emerged as a key feeding and rooting site (staging site) for shorebirds in South Korea. However, there has yet to be any study conducted on the behavior and diet of shorebirds in this region. In this study, we first compared behavior and diet of two representative shorebird species with different morphologies, Bar-tailed Godwits (Limosa laponica) and Great Knots (Calidris tenuirostris) that co-occur in the tidal flat of Yubu Island near the Geum Estuary. During the April to May of 2016, using a camera equipped with a telescope, we recorded 36 individuals during low tide. Behavior was significantly different between the two shorebird species (Chi-square test, P = 0.037), although feeding success rate was similar between the two species (ANOVA, P > 0.05); the Bar-tailed Godwit showed more probings and less peckings than the Great Knot. In addition, Bar-tailed Godwits walked less (ANOVA, P < 0.001) and were less alert (ANOVA, P < 0.005) than Great Knots. On the other hand, diet composition was significantly different between them (Chi-square test, P = 0.010); Both species fed mainly upon Mollusca but the Great Knot fed on noticeably more Annelida (lugworms) than the Bar-tailed Godwit. Among Mollusca, both species consumed more gastropods than bivalves (ANOVA, P < 0.001).Based on these results, it is thought that Bar-tailed Godwits may save energy through less pecking together with less walking and less being alert, since it fed on less food than Great Knots. These results on behavior and diet will be helpful for conservation of the two species in the Geum Estuary tidal flats, a key stopover site of many shorebirds in the East Asian-Australasian Flyway (EAAF)

Whole Genome Analysis of Lactobacillus plantarum Strains Isolated From Kimchi and Determination of Probiotic Properties to Treat Mucosal Infections by Candida albicans and Gardnerella vaginalis

Three Lactobacillus plantarum strains ATG-K2, ATG-K6, and ATG-K8 were isolated from Kimchi, a Korean traditional fermented food, and their probiotic potentials were examined. All three strains were free of antibiotic resistance, hemolysis, and biogenic amine production and therefore assumed to be safe, as supported by whole genome analyses. These strains demonstrated several basic probiotic functions including a wide range of antibacterial activity, bile salt hydrolase activity, hydrogen peroxide production, and heat resistance at 70°C for 60 s. Further studies of antimicrobial activities against Candida albicans and Gardnerella vaginalis revealed growth inhibitory effects from culture supernatants, coaggregation effects, and killing effects of the three probiotic strains, with better efficacy toward C. albicans. In vitro treatment of bacterial lysates of the probiotic strains to the RAW264.7 murine macrophage cell line resulted in innate immunity enhancement via IL-6 and TNF-α production without lipopolysaccharide (LPS) treatment and anti-inflammatory effects via significantly increased production of IL-10 when co-treated with LPS. However, the degree of probiotic effect was different for each strain as the highest TNF-α and the lowest IL-10 production by the RAW264.7 cell were observed in the K8 lysate treated group compared to the K2 and K6 lysate treated groups, which may be related to genomic differences such as chromosome size (K2: 3,034,884 bp, K6: 3,205,672 bp, K8: 3,221,272 bp), plasmid numbers (K2: 3, K6 and K8: 1), or total gene numbers (K2: 3,114, K6: 3,178, K8: 3,186). Although more correlative inspections to connect genomic information and biological functions are needed, genomic analyses of the three strains revealed distinct genomic compositions of each strain. Also, this finding suggests genome level analysis may be required to accurately identify microorganisms. Nevertheless, L. plantarum ATG-K2, ATG-K6, and ATG-K8 demonstrated their potential as probiotics for mucosal health improvement in both microbial and immunological contexts

Profiling age-related epigenetic markers of stomach adenocarcinoma in young and old subjects

The purpose of our study is to identify epigenetic markers that are differently expressed in the stomach adenocarcinoma (STAD) condition. Based on data from The Cancer Genome Atlas (TCGA), we were able to detect an age-related difference in methylation patterns and changes in gene and miRNA expression levels in young (n = 14) and old (n = 70) STAD subjects. Our analysis identified 323 upregulated and 653 downregulated genes in old STAD subjects. We also found 76 miRNAs with age-related expression patterns and 113 differentially methylated genes (DMGs), respectively. Our further analysis revealed that significant upregulated genes (n = 35) were assigned to the cell cycle, while the muscle system process (n = 27) and cell adhesion-related genes (n = 57) were downregulated. In addition, by comparing gene and miRNA expression with methylation change, we identified that three upregulated genes (ELF3, IL1??, and MMP13) known to be involved in inflammatory responses and cell growth were significantly hypomethylated in the promoter region. We further detected target candidates for age-related, downregulated miRNAs (hsa-mir-124-3, hsa-mir-204, and hsa-mir-125b-2) in old STAD subjects. This is the first report of the results from a study exploring age-related epigenetic biomarkers of STAD using high-throughput data and provides evidence for a complex clinicopathological condition expressed by the age-related STAD progression. © the authors, publisher and licensee Libertas Academica Limitedopen